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DETERMINING TRANSMISSION LATENCY 
FIELD 

[0001] Embodiments of the invention relate to transmission latency in networks, 
and more particularly to a system and method for determining transmission latency 
in networks. 

BACKGROUND 

[0002] Computer networking is prevalent amongst many users of computing 
devices, such as personal computers and workstations. Networking allows users 
of computing devices to communicate with each other in various forms, such as 
the execution of a computer program (e.g., a video game), on a computing device 
such as a personal computer, while displaying the results on a separate system 
with a larger display device, such as on a home entertainment system. 

[0003] Though an effective form of communication between devices, networking 
is not without shortcomings. One such shortcoming is in the area of latency due to 
the transmission of data and instructions from one device to another. In certain time- 
sensitive transmissions, such as real-time streaming audio transmissions, the latency 
could result in the audio data becoming asynchronous (out of sync) with data 
displayed on a display device, resulting in an undesirable presentation of an executed 
computer program. As such, efforts have been undertaken to better measure and 
reduce the transmission latencies in networks. 

[0004] Currently, understanding and measuring of the transmission latency 
often requires a break down of the underlying data streaming system to several 
subsystems, then analyzing the latency of each subsystem, and thereafter 
determining (but not really testing) the latency of the entire data streaming system. 
This approach generally proves very costly to implement. For instance, it may not 
be possible to analyze the latency experienced by a data streaming system through 
analysis of the latency experienced by its subsystems. The reason is that there 
may exist difficulties in analyzing the latency experienced by interactions between 
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the subsystems, such as feedback loops between the subsystems. Thus in these 
cases, a breakdown, or white-box analysis, may not be accurate for evaluating the 
latency. 

[0005] In addition, network latency is often regarded as the time period 
between the transmittal of the first byte of a package, such as an audio package, 
by host computing device, and the time that a target device receives the lastbyte 
of the transmitted package. This benchmark measurement may be inaccurate, as 
other factors may need to be included for a more accurate latency measurement. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0006] Embodiments of the invention may best be understood by referring to 
the following description and accompanying drawings that are used to illustrate 
embodiments of the invention. 

[0007] FIG. 1 is a block diagram of a system in which embodiments of the 
invention may be practiced. 

[0008] FIG. 2 is a flow chart illustrating an exemplary process according to 
an exemplary embodiment of the invention. 

[0009] FIG. 3, FIGs. 5A-B and FIGs. 7A-B illustrate exemplary waveforms 
for use with an exemplary embodiment of the invention. 

[00010] FIG. 4 and FIGs. 6A-B are flow charts further illustrating exemplary 
processes according to an exemplary embodiment of the invention. 
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DETAILED DESCRIPTION 

[00011] Embodiments of the invention generally relate to systems and methods 
for determining transmission latencies in a network environment. Herein, one 
embodiment of the invention may be applicable to media devices used in a variety 
of computing devices, which are generally considered stationary or portable 
electronic devices. Examples of a computing device may include, but are not 
limited or restricted to a computer, a set-top box, video game systems, music 
playback systems, and the like. 

[00012] Reference in the specification to the term "one embodiment of the 
invention" or "an embodiment of the invention" means that a particular feature, 
structure, or characteristic described in connection with the embodiment of the 
invention is included in at least one embodiment of the invention. The appearances 
of the phrase "in one embodiment of the invention" in various places in the 
specification are not necessarily all referring to the same embodiment of the 
invention. Some embodiments of the invention are implemented in a machine- 
accessible medium. A machine- accessible medium includes any mechanism that 
provides (i.e., stores and/or transmits) information in a form accessible by a 
machine (e.g., a computer, network device, personal digital assistant, manufacturing 
tool, any device with a set of one or more processors, etc.). For example, a 
machine-accessible medium includes recordable/non-recordable media (e.g., read 
only memory (ROM); random access memory (RAM); magnetic disk storage media; 
optical storage media; flash memory devices; etc.), as well as electrical, optical, 
acoustical or other form of propagated signals (e.g., carrier waves, infrared signals, 
digital signals, etc.), etc. 

[00013] In the following description, numerous details are set forth. It will be 
apparent, however, to one skilled in the art, that the embodiments of the invention 
may be practiced without these specific details. In other instances, well-known 
structures and devices are shown in block diagram form, rather than in detail, in 
order to avoid obscuring the embodiments of the invention. 
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[00014] Also in the following description are certain terminologies used to 
describe features of the various embodiments of the invention. For example, the 
term "digital media adaptor" refers to a peripheral that creates a link between 
personal computers, TVs, and stereos. It can receive digital media from the 
personal computers using wireless networking technologies, and can connect to 
TVs and stereos using standard A/V cables. The term "software" generally 
denotes executable code such as an operating system, an application, an applet, a 
routine or even one or more instructions. The software may be stored in any type 
of memory, namely suitable storage medium such as a programmable electronic 
circuit, a semiconductor memory device, a volatile memory (e.g., random access 
memory, etc.), a non-volatile memory (e.g., read-only memory, flash memory, 
etc.), a floppy diskette, an optical disk (e.g., compact disk or digital versatile disc 
"DVD"), a hard drive disk, or tape. The term "signal processing system" refers to 
electronic circuits that remove information from signals, such as analog signals, as 
quantifiable units for further analysis, such as performance of mathematical 
operations. 

[00015] With reference to FIG. 1, an exemplary embodiment of a system, such as 
an audio/video system 1, is illustrated. In an exemplary embodiment of the invention, 
the audio/video system 1 is a remote gaming system, and comprises a source device 
100, a sink device 102 and a display device 103. More specifically, according to one 
embodiment of the invention, the "sink device" is a device for executing a program, 
such as a personal computer that is able to execute a game program. The sink device 
102 may be a media adaptor while the display device 103 may be a television screen 
for an entertainment system 104. 

[00016] The source device 100 executes a game program, with the execution 
results, such as images and sound, transmitted to the sink device 102 such as by 
wireless transmission 105 and then displayed on the display device 103 of the 
entertainment system 104. In this way, the often larger display device 103 
(compared to a smaller computer screen (not shown) of the source device 100) can 
be used for presenting of the images and sound to users. These transmissions 
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from the source device 100, such as an audio source, to the sink device 102, such 
as an audio sink, however, are prone to end-to-end (source-to-sink) transmission 
latency as symbolically shown by line 106. Embodiments of the invention use a 
latency determination system 1 10 to determine the transmission latency 106 in the 
audio/video system 1. 

[00017] The latency determination system 110 comprises a recordation subsystem 
1 1 1, a pattern insertion subsystem 113 and a signal processing subsystem 1 12 which 
includes a filter subsystem 1 14, a timing subsystem 1 1 5 and latency determination 
logic 1 16, as described in greater detail in conjunction with FIGs. 2-6B. The latency 
determination system 110 may also be implemented within the entertainment system 
104, the sink device 102, source device 100 or in a separate computing device (as 
shown in FIG. 1) in communication with the sink device 102 and the source device 
1 00, such as in another personal computer. 

[00018] FIG. 2 is a flow chart which, in conjunction with FIG. 1 , illustrates an 
exemplary process of an exemplary embodiment of the invention. As shown in FIG. 
2 (following the start block 200), a predetermined pattern 1 17 is inserted in signals 
130a and 130b (block 210) to form a pattern-inserted data stream 130. Signals 130a 
and 130b thus each comprise the original data stream 121 and the predetermined 
pattern 117. The pattern-inserted data stream 130 is then forwarded to source 
output 140 and to the sink device 102 in forms of signals 130a and 130b, each 
containing the same original data stream 121 and the same predetermined pattern 
1 17. Alternatively, the predetermined pattern 1 17 can also be inserted separately 
into copies of the original data stream 121 to separately form signals 130a and 130b, 
which are then forwarded to the sink device 1 02 and to the source output 140, 
respectively. In an exemplary embodiment of the invention, the signal 130a is 
received in a digital media adaptor (DMA) 150 of the sink device 102 and then sent 
to the sink output 154, as well as to the display device 103. 

[00019] Next, signals 130a and 130b are received in the latency determination 
system 110 from the sink output 154 and the source output 140, respectively (blocks 
220, 230). In an exemplary embodiment of the invention, the signals 130a and 130b 
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are first received in a recordation subsystem 1 1 1 prior to their receipt in the signal 
processing subsystem 1 12, as described in greater detail in conjunction with FIG. 4 
below. The signal processing subsystem 1 12 then determines a transmission 
latency 106 between the received signals 130a and 130b based on the predetermined 
pattern 117 (block 240), as described in greater detail in conjunction with FIGs.6A- 
B below. The overall process then ends (block 250). 

[00020] FIG. 3 illustrates an exemplary predetermined pattern 1 1 7 for use with 
the process of FIG. 2 (block 210). In an exemplary embodiment of the invention, 
the predetermined pattern 1 17 is a substantially sinusoidal waveform as 
represented in FIG. 3 using the x and y axes of the Cartesian coordinate system. 
The predetermined pattern 1 17 comprises a predetermined period 301 that is 
greater than a transmission latency period 106. In the example shown in FIG. 3, 
the waveform of the predetermined pattern 117 follows the exemplary Equation 1: 

y - ((100-x/tc)/100)*cos(x) (Equation 1) 

where "x" may range from O.O to 100*te (k =3.1415926.. .) and may be 
incremented at a predetermined interval such as 0.01 for example. As shown if 
FIG. 3, according to one embodiment of the invention, the predetermined pattern 
117 generated by following Equation 1 has a finite period 301 of 100*7i. Suitably, 
period 301 is greater than transmission latency period 106. 

[00021] For example, if a transmission latency period 106 acceptable to the 
audio/video system 1 is less than 500 milliseconds (ms), then the waveform of the 
predetermined pattern 1 17 is predetermined to have a period 301 of for example 
700ms or larger. As shown in FIG. 1, the latency determination system 110 
comprises a pattern insertion subsystem 1 1 3 which can generate and insert the 
predetermined pattern 117 into signals 130a and 130b. Suitably, the latency 
determination system 1 10 is programmable to generate different predetermined 
patterns 117 based on differently programmed instructions, such as different wave 
pattern equations. 
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[00022] FIG. 4 is an exemplary flow chart, which in conjunction with FIG. 5A, 
which illustrates an exemplary process for receiving the signals 130a and 130b 
shown in FIG. 2 (blocks 220, 230). As shown in FIG. 4, following the start (block 
400), the received signals 130a and 130b are recorded in a combination waveform 
500 (block 410) as shown in FIG. 5 A. The transmission latency 106 between the 
received signals 130a and 130b is then determined by the signal processing 
subsystem 112 from the combination waveform 500 as described below and in 
greater detail in conjunction with FIG. 6A-B. The process is then returned (block 
420) to FIG. 2 (block 230). As shown in FIG. 1, the latency determination system 
1 10 comprises a recordation subsystem 111, having inputs (not shown) to receive 
the signals 130a and 130b. The recordation subsystem 1 1 1 then records signals 
1 30a and 1 30b into the combination waveform 500. 

[00023] FIG. 5A illustrates an exemplary combination waveform 500 for use 
with an exemplary embodiment of the invention shown in FIG. 4. As shown in FIG. 
5 A, the combination waveform 500 comprises a recording 501a of the signal 130a 
and a recording 501b of the signal 130b. The recordings 501a and 501b are then 
used by the signal processing subsystem 1 12 of FIG. 1 to determine the 
transmission latency 106 between the received signals 130a and 130b as described 
below and in greater detail in conjunction with FIG. 6A-B. Suitably, the 
combination waveform 500 is stored in a wave (.wav) file. 

[00024] FIG. 6A is a flow chart, which in conjunction with FIG. 7A, further 
illustrates an exemplary process shown in FIG. 2 (block 230) in which the signal 
processing subsystem 1 12 determines the transmission latency 106 between the 
signals 130a and 130b. As shown in FIG. 6A, following the start (block 600), a 
pattern 700a (FIG. 7 A) corresponding to the predetermined pattern 1 17 is 
obtained from signal 130a (block 610). In an exemplary embodiment of the 
invention, the pattern 700a is obtained from the recording 501a of the signal 130a 
(shown in FIG. 5 A). Next, a pattern 700b (FIG. 7A) corresponding to the 
predetermined pattern 1 17 is obtained from signal 130b (block 620). In an 
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exemplary embodiment of the invention, the pattern 700b is obtained from the 
recording 501b of the signal 130b (shown in FIG. 5A). As shown in FIG. 7A, 
patterns 700a and 700b have substantially the same pattern character as that of the 
predetermined pattern 1 17 but which differ somewhat from the predetermined 
pattern 1 17 due to inclusion of other signals such as noise during the transmission, 
and insertion processes. 

[00025] Next, in FIG. 6A, time-positions 702 and 701 (FIG. 7A) corresponding 
to the obtained patterns 700a and 700b, respectively, are determined (blocks 640, 
650). As shown in FIG. 7A, in an exemplary embodiment of the invention, time- 
positions 702 and 701 correspond to the start of the period for the patterns 700a 
and 700b, respectively. 

[00026] The time-positions 702 and 701 are then used in FIG. 6 A to determine a 
latency value between time-positions 702 and 701 (block 660), which is the 
transmission latency 106. As show in FIG. 7A, the time-position 702 for the signal 
130a is at a later time than the time-positions 701 for the signal 130b, the difference 
of which is the latency value represented by dl . This is due to the transmission 
latency 106 caused by transmission of the signal 130a from the source device 100 to 
the sink device 102. For example if the start time (e.g., time position 702) of pattern 
700a is at 68ms, and the start time (e.g., time position 701) of pattern 700b is at 22ms, 
then the latency value represented by dl is determined to be 46ms (i.e. 68ms-22ms). 
The process is then returned (block 670) to FIG. 2 (block 230). 

[00027] As shown in FIG. 1 , the latency determination system 1 1 0 comprises a 
signal processing subsystem 1 12 which includes a filter subsystem 1 14 for performing 
the operations of FIG. 6A (blocks 610 and 620) for filtering out patterns 700a and 700b 
corresponding to the predetermined pattern 117, from the signal 130a and 130b, 
respectively. The signal processing subsystem 112 also includes a timing subsystem 
1 14 for performing the operations of FIG. 6A (blocks 630 and 640) for determining 
timing time-position 702 and 701 corresponding to the obtained patterns 700a and 700b, 
respectively, from the patterns 700a and 700b, respectively. The signal processing 
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subsystem 1 12 also includes latency determination logic 1 16 for performing the 
operations of FIG. 6A (block 660) for determining a latency value based on the time- 
positions 702 and 701 . The latency determination logic 116 may be implemented in 
hardware or software stored on a memory storage medium (not shown). 

[00028] In an exemplary embodiment of the invention, the latency determination 
system 110 shown in FIG. 1 can also determine a latency average by determining a 
plurality of latencies for signals 130a and 130b. In an exemplary embodiment of 
the invention, latency determination system 1 1 0 periodically samples signals 130a 
and 130b and applies the above-described process of FIG. 2 to each sample, to 
determine a latency value for each sample. In this exemplary embodiment of the 
invention, the operations of the recordation subsystem 111 described in FIG. 4 is 
applied to each received sample of the signals 130a and 130b, with each sample 
recorded in a combination waveform 510 as shown in FIG. 5B. As shown in FIG. 
5B, in this exemplary embodiment of the invention, the combination waveform 510 
comprises a plurality of recordings 510a, such as 510a 1 and 510a2, with each 
recording corresponding to a different sampling of the signal 130a at the time (t) the 
signal 130a was sampled. Likewise, a plurality of recordings 510b, such as 51 Obi 
and 5 10b2, is made, with each recording corresponding to a different sampling of 
the signal 130b, at the time (t) the signal 130b was sampled. 

[00029] The process described in FIG. 6A is then applied to the recordings of 
each sample, such to recordings 510al and 51 Obi, then to recordings 510a2 and 
510b2, etc. Applying the above-described processes of FIG. 6 A (blocks 610, 620) 
to each sample results in set of a patterns 710a (FIG. 7B), such as 710al and 710a2 
corresponding to the predetermined pattern 1 17 to be obtained from samples of 
signal 130a. Likewise, another set of patterns 710b, such as 71 Obi and 710b2, 
corresponding to the predetermined pattern 117 can be obtained from samples of 
signal 130b. In an exemplary embodiment of the invention, the patterns 710a and 
710b are obtained from the recordings 510a and 510b of the samples of signals 130a 
andl30b, respectively (shown in FIG. 5B). Applying the above-described processes 
of FIG. 6 A (blocks 640-660) to each sample results in the determination of time- 
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positions for each of the samples, such as time-positions 711, 722, and 721, 722, 
respectively, and the subsequent determination of a latency value for each sample, 
such as latency values d2 and d3, respectively. 

[00030] Referring to FIG. 6B, following the start (block 680), an average latency 
value can then be determined from the determined latency values for each sample, 
such as from latency values d2 and d3 (block 690). Other useful mathematical 
operations such as calculation of variance, etc may also be performed if desired. 
The process is then returned (block 695) to FIG. 2 (block 230). 

[00031] The latency determination system 1 10 of the invention shown in FIG. 
1 can also process signals that are transmitted in the stereo format. In this 
embodiment of the invention, the above-described processes of FIGs. 2-7B are 
performed once for the left channel of the stereo transmission to determine a left- 
channel delay, and again for the right channel to determine a right-channel delay. 

[00032] In an exemplary embodiment of the invention, the software that, if 
executed by the latency determination system 110, will cause the latency 
determination system 1 10 to perform the above operations described in conjunction 
with FIGs. 2-7B is stored in a storage medium, such as in a main memory (not 
shown), and storage devices (not shown). 

[00033] It should be noted that the various features of the foregoing 
embodiments of the invention were discussed separately for clarity of description 
only and they can be incorporated in whole or in part into a single embodiment of 
the invention having all or some of these features. 



